Chapter 1: Concepts and definitions

Goals

Understand conceptual and operational definitions

Know some strategies for clarifying conceptual definitions and writing conceptual definitions

Defining terms

Research Questions

Does religion play a role in civil wars?
What causes increasing income inequality in least developed countries?
Why has public opinion on same sex marriage liberalized so rapidly?
Is the U.S. becoming more polarized?

Not everyone is going to agree on what these terms mean!

Before we can get anywhere researching these questions, we need to define the thing we’re studying

Defining things: harder than it seems!

Terms

Conceptual Definition: A description of the concrete, measurable properties of a concept and the unit of analysis to which it applies
Unit of Analysis: The entity that that is being studied. For instance: individuals, governments, parties etc.
Operational Definition: A description of the instrument used to measure the concept

Crafting Conceptual Definitions: Unit of Analysis

What is the unit? What entity possesses the characteristic?
- The U.S. and Canada are democracies (concept: democracy, unit: countries)
- Lincoln, LBJ, and Trump are the three tallest U.S. presidents (concept: height, unit: presidents/people)
- Labour is the UK’s main center-left political party (concept: party ideology/family, unit: political party)

Crafting Conceptual Definitions: Key features

What are the essential features of the concept?
- If there are cases or definitions everyone agrees on, what characteristics do they share?
- Are certain qualities necessary or sufficient for a case to belong to a category?
- What characteristics are most distinctive to those cases?
- What is most helpful for clarifying edge-cases?
Are there multiple dimensions or just one? If two characteristics always occur together, you might only need to account for one of them!

Crafting Conceptual Definitions: transparency

We want to avoid being Potter Stewart:

So measurable and transparent definitions are key.

Some steps to take:

Identify the unit of analysis
Make a list of important properties, clear examples and non-examples, and/or generally accepted definitions
Eliminate or refine characteristics that aren’t distinctive.
Remove items that aren’t measureable
Reduce dimensions where possible
Refine as needed

Conceptualizing Democracy

Some proposed features:

Regular elections with meaningful alternatives
Peaceful transfer of power
Free expression
A competitive media environment
Autonomous political groups (political associations, unions, interest groups etc.) that can pressure the government
Rule of law
Checks and balances
Property rights

Minimalists might argue that 1 and 2 alone are necessary and sufficient. Others may argue that even these 8 are oversimplifying.

Operational Definitions

Operationalization

Even where we agree on a definition, we will need to measure a concept and there is often slippage here

Considerations: Parsimony

In many cases, we trade some truth for simplicity

This is a very accurate rendering of the DC metro area

But this is probably more useful for getting around.

Considerations: Parsimony

Similarly: we might consider “democracy” to be a highly complex multidimensional concept

mindmap
  root((Democracy))
    Liberal Rights
        Free Speech
        Free Association
        Rule of law
    Deliberation
        Respectful Dialogue
        Adversarial Press
        Broad Participation
    Egalitarianism
        Equal Rights
        Equitable access to resources
        Diverse representation
    Majoritarianism
        Fair elections
        Peaceful transfer of power
        Meaningful alternatives

Considerations: Parsimony

…but we might still prefer an operational definition that relies on a subset of concepts that are easy to measure.

mindmap
  root((Democracy))
    Majoritarianism
        Fair elections
        Peaceful transfer of power
        Meaningful alternatives

Operationalizing Democracy

Dictatorship and Democracy (Alvarez, Cheibub, Limongi, & Przeworski 1996)

Dichotomous measure. Democracies have the following:

Have popular elections to fill seats
Have more than one party
The incumbent sometimes loses

Varieties of Democracy (V-DEM) Project (link)

Democracy along multiple dimensions. Scores are determined by:

Polling a large group of area experts on a range of democratic dimensions (electoral, deliberative, liberal etc.)
Using an algorithm to aggregate those responses into scores along each dimension.

Both of these approaches have acknowledged trade-offs. The minimalist view has the advantage of parsimony, but the more nuanced measures may get us closer to the ideal conceptual definition.

But neither is free from error.

Also note, these are influenced by different conceptual definitions, which are in turn potentially influenced by some normative considerations.

Considerations: Reliability

Reliability refers to how consistently the same measurement instrument produces the same result

Test-Retest method: asking the same questions to a new random sample should produce roughly the same result
Split half: for a measurement using a multi-item scale, we can split results and use half the questions to predict the remaining half.

(One purported advantage of the minimalist definition of democracy is that its relatively stable because it allows less room for subjectivity.)

Considerations: Validity

Validity refers to whether an operational definition matches the concept we’re interested in.
A challenge to validity might come from cases seem miscategorized in our definition. For example: the minimalist definition of democracy suggests that Japan was authoritarian for much of the latter half of the 20th century.

Measurement Error

Random Error
- Interference
- Data entry errors
- Random non-response
Systematic Error
- Hawthorne effect
- Social desirability bias
- Systematic non-response (surveys) or missing data (everything)

Measurement Error

Measurement error is a source of noise, systematic measurement error is a source of bias
We’ll generally find that random noise is much easier to deal with than bias because we can simply collect more data. Bias, on the other hand, presents profound challenges to research because we can only correct it if we can measure it

Measuring Party ID

Consider measuring party identification:

Instrument	Problem
Registration	Excludes new/non-voters, people in states without partisan registration, and people who register one way and vote another
Policy views	Complicated to measure, and a surprising number of people don’t have consistent policy views!
Voting Behavior	Excludes new/non-voters, may not be consistent even in a single election, people may exaggerate or forget voting behavior.
Self Description*	Subjective, far more self-described “independents” than people who consistently vote that way

Party ID

The ANES method for measuring party ID is hardly the only option we have, but it has some nice features:
- Its relatively reliable and survey respondents understand it
- Its close to the conceptual definition of party ID as a sort of self-affiliation
- Its transparent and widely used
- It limits the influence of social desirability bias by giving respondents lots of options
- Its linked to observed behavior: vote choice

Key points

Asking social science questions requires us to define complex ideas and measure them. And we can lose a lot in that process.
There are some wrong answers, but there is generally no single correct answer. We generally have to make trade-offs between things we might value like:
- Parsimony
- Reliability
- Validity
- Probability of random or systematic errors
We can’t really achieve perfection, but where there is disagreement, we want to be transparent about methods and limitations.

Reminders

Install R and R-Studio for Friday discussion
Start thinking about questions for pilot design.